Recently, directly using large language models (LLMs) has been shown to be the most reliable method to evaluate QA models. However, it suffers from limited interpretability, high cost, and environmental harm. To address these, we propose to use soft exact match (EM) with entitydriven answer set expansion. Our approach expands the gold answer set to include diverse surface forms, based on the observation that the surface forms often follow particular patterns depending on the entity type. The experimental results show that our method outperforms traditional evaluation methods by a large margin. Moreover, the reliability of our evaluation method is comparable to that of LLM-based ones, while offering the benefits of high interpretability and reduced environmental harm.

The recent development of Monkeypox disease among various nations poses a global pandemic threat when the world is still fighting Coronavirus Disease-2019 (COVID-19). At its dawn, the slow and steady transmission of Monkeypox disease among individuals needs to be addressed seriously. Over the years, Deep learning (DL) based disease prediction has demonstrated true potential by providing early, cheap, and affordable diagnosis facilities. Considering this opportunity, we have conducted two studies where we modified and tested six distinct deep learning models-VGG16, InceptionResNetV2, ResNet50, ResNet101, MobileNetV2, and VGG19-using transfer learning approaches. Our preliminary computational results show that the proposed modified InceptionResNetV2 and MobileNetV2 models perform best by achieving an accuracy ranging from 93% to 99%. Our findings are reinforced by recent academic work that demonstrates improved performance in constructing multiple disease diagnosis models using transfer learning approaches. Lastly, we further explain our model prediction using Local Interpretable Model-Agnostic Explanations (LIME), which play an essential role in identifying important features that characterize the onset of Monkeypox disease.

We introduce a new open information extraction (OIE) benchmark for pre-trained language models (LM). Recent studies have demonstrated that pre-trained LMs, such as BERT and GPT, may store linguistic and relational knowledge. In particular, LMs are able to answer ``fill-in-the-blank'' questions when given a pre-defined relation category. Instead of focusing on pre-defined relations, we create an OIE benchmark aiming to fully examine the open relational information present in the pre-trained LMs. We accomplish this by turning pre-trained LMs into zero-shot OIE systems. Surprisingly, pre-trained LMs are able to obtain competitive performance on both standard OIE datasets (CaRB and Re-OIE2016) and two new large-scale factual OIE datasets (TAC KBP-OIE and Wikidata-OIE) that we establish via distant supervision. For instance, the zero-shot pre-trained LMs outperform the F1 score of the state-of-the-art supervised OIE methods on our factual OIE datasets without needing to use any training sets. Our code and datasets are available at https://github.com/cgraywang/IELM

This paper shows how to construct knowledge graphs (KGs) from pre-trained language models (e.g., BERT, GPT-2/3), without human supervision. Popular KGs (e.g, Wikidata, NELL) are built in either a supervised or semi-supervised manner, requiring humans to create knowledge. Recent deep language models automatically acquire knowledge from large-scale corpora via pre-training. The stored knowledge has enabled the language models to improve downstream NLP tasks, e.g., answering questions, and writing code and articles. In this paper, we propose an unsupervised method to cast the knowledge contained within language models into KGs. We show that KGs are constructed with a single forward pass of the pre-trained language models (without fine-tuning) over the corpora. We demonstrate the quality of the constructed KGs by comparing to two KGs (Wikidata, TAC KBP) created by humans. Our KGs also provide open factual knowledge that is new in the existing KGs. Our code and KGs will be made publicly available.

This paper c onsider s the problem of dynamic task allocation, where tasks are unknowingly distributed over an environment. We aim to address the multi - robot exploration aspect of the problem, while solving the task - allocation aspect. To that end, we first propose a novel nature - inspired approach called "hunter and gatherer". W e consider each task comprised of two sequential su btasks: detection and completion, where each subtask can only be carried out by a certain type of agent. Thus, this approach employs two complementary teams of agents: one agile in detecting (hunters) and another dexterous in completing (gatherers) the tasks. Then, we propose a multi - robot exploration algorithm for hunters and a multi - robot task allocation algorithm for gatherer s, both in distributed manner and based on innovative notions of "certainty and uncertainty profit margins". Statistical analysis on simulation results confirm the efficacy of the proposed algorithms. Besides, it is statistically prove n that the proposed s olutions function fairly, i.e. for each type of agent, the overall workload is distributed equally. I. Introduction Multi - robot systems are expected to complete tasks that are unfeasible, laborious or inefficient for a single agent to accomplish [1] . Employing multi - robot systems entails addressing various problems on the subject of task allocation [2], exploration [3], coordination [4], learning [5], and heterogeneity [6] . Among all these problems, the problem of multi - robot task allocation (MRTA), assign ing a group of tasks to individual robots, is the most deep - seated problems of multi - robot systems, where its complexity increases considerably by a wide variety of factors. Regarding, a MRTA problem where tasks are unknowingly distributed over an environment needs to be addressed by solving the problem from both MRTA and multi - ro bot exploration perspectives. This problem can even get more complicated if each task is divided into two sequential subtasks and each subtask can only be carried out by a certain type of agent.

Consider a dynamic task allocation problem, where tasks are unknowingly distributed over an environment . This paper considers ea ch task comprised of two sequential subtasks: detection and completion, where e ach subtask can only be carried out by a certain type of agent . We address th is problem using a novel natur e - inspired approach called "hunter and gathere r" . Th e proposed method employs two complementary teams of agents: one agile in detecting (hunters) and another dexterous in completing (gathere r s) the tasks . To minimize the collective cost of task accomplishments in a distributed manner, a game - theor etic solution is introduced to couple agents from complementary teams . We utiliz e market - based negotiation models to develop incentive - based decision - making algorithms rely ing on innovative notions of " certainty and uncertainty profit margins " . The simulation results demonstrate that employing two complementary teams of hunters and gatherers can effectually improve the number of tasks completed by agents compared to conventional methods, while the collec tive cost of accomplishments is minimized . In addition, t he stability and efficacy of the proposed solutions are studied using Nash equilibrium analysis and statistical analysis respectively . It is also numerically show n that the proposed solution s function fairly, i.e. for each type of agent, the overall w orkload is distributed equally . Index Terms -- Distributed multiagent system, dynamic task allocation, game theory, negotiation. Multirobot systems are expected to undertake imperative roles in a wide variety of fields such as urban search and rescue (USAR) [1, 2], agricultural field operations [3], security patrols [4, 5], environmental monitoring [6], and industrial procedures [7] . Studies have shown that multi - robot systems have advantage over single - robot systems by offering more reliability, redundancy, and time efficiency when the nature of the tasks is inherently dist ributed [8] . Nonetheless, the problem of multi - robot task - allocation (MRTA) poses many critical challenges that has called for investigation in the past two decades [9 - 11] . In this regards, t he complexity of MRTA problems increases significantly in a dynamic environment, where the number and location of tasks are unknown for agents [12, 13] . Thus, robot s need to explore the environment to find tasks before accomplishing them.

While machine learning has achieved remarkable results in a wide variety of domains, the training of models often requires large datasets that may need to be collected from different individuals. As sensitive information may be contained in the individual's dataset, sharing training data may lead to severe privacy concerns. Therefore, there is a compelling need to develop privacy-aware machine learning methods, for which one effective approach is to leverage the generic framework of differential privacy. Considering that stochastic gradient descent (SGD) is one of the mostly adopted methods for large-scale machine learning problems, two decentralized differentially private SGD algorithms are proposed in this work. Particularly, we focus on SGD without replacement due to its favorable structure for practical implementation. In addition, both privacy and convergence analysis are provided for the proposed algorithms. Finally, extensive experiments are performed to verify the theoretical results and demonstrate the effectiveness of the proposed algorithms.

Almost exactly a year ago, in April 2010, professional motocross rider Randy Childers sustained serious injuries after a crash in the last race of the day at Cowboy Badlands in West Beaumont, Texas. He suffered broken ribs and a fractured wrist, but most seriously a crushed vertebra in his neck (C3) that required him to be airlifted to Houston, where surgeons inserted an artificial vertebra and fused two others together (C4 and C5) during a marathon operation that lasted 12 hours. Today, the 24-year-old is the star in a single-patient trial of Rice University's RiceWrist robot, a wearable exoskeleton that mimics the joints from his shoulder to his hand. After months of traditional physical therapy, Childers had recovered enough by October to walk (albeit slowly) into the basement lab at Rice and begin to use the RiceWrist, which is built to reconnect motor pathways in the brain through repetitive movement. After just two weeks, Rice Professor Marcia O'Malley says, Childers was doing most of the work himself.